<html>
<head><meta charset="utf-8"><title>Structured Code Generation · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html">Structured Code Generation</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="206559121"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206559121" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> theduke <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206559121">(Aug 11 2020 at 10:15)</a>:</h4>
<p>I've been dissatisfied with the current state of code generation for a while. </p>
<p>The prevailing options are either using <code>syn</code> + <code>quote</code>, or - as RA is doing - basic string concatentation.</p>
<p>Both allow creating malformed output which can only be detected at runtime,  and limit generation to a simple single pass, unless you either have your own tailored AST, or deal with manipulating the <code>syn</code> AST, which is very awkward, since it really isn't built for manipulation.</p>
<p>A solution would be a library tailored for code generation. It could either use a custom AST for just that purpose, or provide a convenient wrapper around existing ones, like <code>ra_syntax</code>.</p>
<p>Some prior art: <a href="http://www.lst.de/~cs/kode/libkode.html">Kode</a> for C++,  <a href="https://github.com/nette/php-generator">php-generator</a>.</p>
<p>I'm curious if there is interest in something like this from the rust-analyzer side.</p>



<a name="206563872"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206563872" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206563872">(Aug 11 2020 at 11:27)</a>:</h4>
<p>I think we have the beginnings of this in <a href="https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/ra_syntax/src/ast/edit.rs">edit.rs</a> and <a href="https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/ra_syntax/src/ast/make.rs">make.rs</a></p>



<a name="206564168"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206564168" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206564168">(Aug 11 2020 at 11:31)</a>:</h4>
<p>there's obviously a lot to be improved there, but I think the general shape (creating and manipulating <code>ast</code> nodes) is what we want</p>



<a name="206564230"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206564230" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206564230">(Aug 11 2020 at 11:32)</a>:</h4>
<p>note that we need to be able to incorporate user-written code as-is</p>



<a name="206564277"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206564277" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206564277">(Aug 11 2020 at 11:33)</a>:</h4>
<p>I'm pretty sure matklad has a lot more to say about it though, but he's on vacation currently as far as I know</p>



<a name="206675442"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206675442" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206675442">(Aug 12 2020 at 09:40)</a>:</h4>
<p>Yeah, syntax tree manipulation is a thorny problem, API-wise. </p>
<p>For IDE use-case, I am pretty sure that "creating malformed output" is a feature -- sometimes you need to produce bad output (if parts of it are malformed sytnax nodes from input), and I don't think I've faced a log of bugs which could have been prevented by stronger types in the AST.</p>
<p>For creating &amp; manipulating AST nodes, I indeed like the general shape of <code>make.rs</code> and <code>edit.rs</code>, I think the API is optimal for immutable syntax trees. The impl is suboptimal, in two ways:</p>
<ul>
<li>it is hand-written, we should investigate deriving it from <a href="https://github.com/rust-analyzer/ungrammar/blob/master/rust.ungram">https://github.com/rust-analyzer/ungrammar/blob/master/rust.ungram</a></li>
<li>it uses <code>parse . to_string</code> on the lowest layer, but we should switch to just assembling the trees directly</li>
</ul>



<a name="206675598"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206675598" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206675598">(Aug 12 2020 at 09:43)</a>:</h4>
<p>However, and that is really tantalizing, the most usable API for editing syntax trees is a mutable one.  Ie, when you can have reference to several nodes in the same tree, and modifications via one reference are reflected in another reference. This gives a much more intuitive API, as you can do things like:</p>
<div class="codehilite"><pre><span></span><code><span class="k">for</span><span class="w"> </span><span class="n">child</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">node</span><span class="p">.</span><span class="n">children</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">child</span><span class="p">.</span><span class="n">remove</span><span class="p">()</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>


<p>I still have hope that we can reconcile rust-analyzer's immutable trees (immutability is the right choice for everything else) and mutable editing API, but, so far, I haven't found  a way to do this.</p>



<a name="206675708"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Structured%20Code%20Generation/near/206675708" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Structured.20Code.20Generation.html#206675708">(Aug 12 2020 at 09:45)</a>:</h4>
<p>Separately, for the task of pragramtic code generation, I think <code>quote!</code> + shelling out to <code>rustfmt</code> (what we do to generate sytnax trees themselves) is a near-optimal choice. You <strong>could</strong> add all kinds of conveniences &amp; type safety there, but you shouldn't as for such tasks "stupid &amp; fast to compile" is a much more important property than "a perfectly cut gem made of types".</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>